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ABSTRACT 

. This report presents evaluation results of an 

Oklahoma City Public Schools bilingual and multicultural education 
program to increase achievement levels and improve multicultural 
attitudes among Spanish speaking elementary school children of 
limited English proficiency. Results of English and Spanish tests of 
language proficiency in kindergarten to grade 2 who were tested on 
the acquisition of basic concepts considered to be prerequisites for 
language learning similarly showed significant performance gains. 
Teacher ratings indicated increases in students 9 positive attitudes 
toward their own culture and toward other cultures. Staff responses 
to questionnaires on the quality of inservice training and parent 
perceptions of classes in English as a Second Language for parents 
were generally favorable. In addition, parents expressed satisfaction 
with the bilingual education program for their children. Principals 
of the schools in which the program was implemented recommended that 
the program be continued, that new and updated materials be provided, 
and that further inservice training for teachers and aides be 
conducted. (Author/MJL) 
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EXECUTIVE SCM4AKY 

Oklahoma City's Title VII Bilingual Education Program is funded under 
the Federal Elementary and Secondary Education Act (P.L. 92-318) . The pri- 
mary purpose of the Bilingual Education Act has berm to establish equal edu- 
cational opportunity for all children, through the establishment and oper- _ 
ation of education programs using bilingual education practices and methods. 
S pecifically , the intent is to provide for children of limited English pro- 
ficiency, classroom instruction designed to enable them, while using their 
native language, to achieve ocmpetency in ±e English language. 

The Oklahoma City Public Schools Bilingual pr o gr a m began in January of 
1977, in three target elementary schools (Westwood, Riverside, and Shidler) , 
which have high, concentrations of Spanish speaking students. For the current 
grant period (1980-81) , a fourth school (Iongfellcw) has been added to the 
progxatn* 

Generally, the goal of the Oklahoma City Public Schools Bilingual Title 
vn Program continues to be to provide an effective program of bilingual/ 
multicultural education which is designed primarily to increase the level 
of achievement for children of limited English speaking ability. The grant 
provides four areas of measurement (objectives) , considered to oe important 
in <3etermining the progress made by these limited English proficiency (LEP) 
students in the following areas: (1) English comprehension, (2) Spanish 
vocabulary and comprehension, (3) Mastery of basic concepts, which are pre- 
sumd to be prerequisites for language learning (and therefore predictors 
of later school success) , and (4) Multicultural classroom appreciation. 
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Results 

The IAS test was utilized to determine the gains in English and Spanish 
oral linguistic proficiency by participating students in grades K-5. Pre- 
test results revealed, as would be expected, that a much larger percentage 
of students began the year with lower carpetency levels in asglish than 

* _ 

Spanish. Within the cons tr aints noted, the abjective of raising 70% of 
these IZP students 1 oral proficiency in English by a full level 
was met, as approximately 79% made a one level (standard deviation) gain (or 
better) during the year. In analyzing the very lc** oatpetency students 1 T*in 
during the year, highly significant (p <C001) gains were noted at e&d grade 
level (K-5) , as well as substantially larger raw score mean gains, than the 
overall population. In all grades (K-5) , for all groups, the mean gains in 
Qaglish oral proficiency were higidy significant (at least p-C .002) for all 
groups of students. 

The same students' performance on IAS Spanish oral proficiency was not 
as substantial as t heir English performance, yet in most cases (and most 
grades) they produced significant gains. Within limitations, approximately 
44% (less than the 70% specified by Objective 1) of the students showed a one 
level (standard deviation) gain or better during the year. Basically, only 
at grade 2 and grade 5 did they not show significant (at least p <.02) mean 
gains between pre and posttesting. This lower percentage of gains in Spanish 
appears to be a reflection of the arphasis of the Title VII program toward 
promoting English acquisition by these LEP students, in an attempt to make 
them ncre able to ocn^ete and excel in English speaking classrooms. It is 
i i iport an t to note, that although the students did not improve as dramatically 
cn the Spanish portion of tte LAS, as they did an the English portion, sub- 
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stantial gains in Spanish proficiency were reported. This is supported by 
the data from the Pr**™ de lectura, Serie Interamerica results, which showed 
that students' vocabulary and apprehension reading skills trains in Spanish, 
were equal to or higher than, the national norming samples for almost every 
grade. For each grade," first through fifth (except 3rd grade, which had 
a slight NCE drop) the NCE gains were greater than zero, and overall, for 
all students, across all grades, showed an NCE gain of +1.-14 (exceeding Ob- 
jective 3) . Any NCE gain greater than zero indicates that progress was 
made in excess of predicted progress based n national norms for the year. 
For the early elementary students (grades K-?) , the results of the 
' Boeftn Test of Basic Concepts showed that overall, approximately 80% made 
gains during the year (thereby meeting Objective 2) . The BTBC reportedly 
measures acquisition of basic concepts, which are presumed to be orerequi- 
sites for language learning., and therefore predictors of later school suc- 
cess. Further, regardless of whether the student's dominant language was 
English or Spanish, the mean raw sccre gains in each grade (K-2) were highly 
significant (most less than P4..001) for the year. 

Finally, although teacher ratings are totally subjective, they indicate 
that Object^ > 4' was met, since the Bilingual teachers and aides rated 87.5% 
of the Title VII students as having a positive attitude toward their own, 
as well as the other cultures represented in their classmoms (an increase 
of 3% over their rating at the start of the year) . 

Participants' Perceptions of the Program 
School Adninistrators. The principals at the school sites all report-** 
ed they had attended inservice training activities presented by the Title 
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VU program. All believed that the Title VH staff made an effort to provide 
information about the purposes and services available through the program, 
aod their Title VU building staff regularly attempted to transnit inf or- 
.nation to than concerning problems or progress in working with the students. 
They also felt that the aides were invaluable in providing an interface be- 
tween the staff and the Spanish speaking students, and their Spanish speaking 
parents. When asked for reocrnnendaticns for improvement, they unanimously 
stated the program should be continued, with emphasis on new and updated 
mate ri als, and further inserviae and training for teachers and aides at the 
building level. 

Parents. A total of 102 parents fron the four target schools responded 
to the Title VU parent survey. Of these -102, 91% stated they had received 
information (in both Spanish and Baglish) explaining both the purpose and 
goals of the Oklahoma City Public Schools Bilingual Program. Most (80-90%) 
parents also reported that ttey had been notified by their child's school 
throughout the year, (again in Biglish and Spanish) of occurrences such as 
r ep ort cards, parent conference r epo rt s, notes from school, necessary im- 
munization information, and parent committee meetings. 

Additionally , irost (over 90%) of the parents felt that their child had 
benefited from the Bilingtial Education program , and they expressed the impor- 
tance to them that their children were being taught both English and Spanish, 
as they wanted their children to continue their bilingual capabilities. Most 
of the parents volunteered that they had witnessed inprovenents in their 
children's progress in school, and are pleased with the program. A large 
matter of them a±so cutmented that they were very grateful for the schools 1 
concern and understanding, and appreciated receiving needed information in 

iv 




ERIC 



Spanish. Not a single parent stated that they felt the program should be 
discontinued. 

Staff training and perceptions. The Title VH staff attended inservice 
training and workshops throughout the year, ranging in duration from 1-2 
days, to a 2-week pre-school workshop. Between 75-100% on each occasion 
rated the q ualit y of the workshop "good" or "very good" . Roughly the same 
percentages felt the enphasis of the workshops were directly applicable to 
their work, and most materials and topics were relevant and necessary. The 
majority of individuals reported each time that more training and assistance 
would be useful, and suggested future topic or training areas included: 
(1) more training to teach Spanish, (2) introduction of new materials, (3) 
more application of ESL in science, math and literature, (4) more training 
in arts, crafts, and games, and (5) more samples of visual presentations 
for ESL. 

Parent English Classes . A final aspect of the Bilingual program invol- 
ved ESL training for parents of Title VTI students, weekly at the Capitol 
Hill Library. The classes were conducted in the evening, and approximately 
50 parents were "regular" participants. The parents responded that they 
basically liked everything about the classes, and would make no recommen- 
dations far iaprovement. The majority of them (over 80%) felt the classes 
had helped than in their everyday communicatio n s , and had helped than in 
tlwir jobs. Most felt they needed to learn mare, and wanted to continue. 

a mary . The vast majority of responses by all participants and 
pare in the surveys were favorable. School administrators, staff, and 
parents felt the program \3s beneficial to the students, enabled them to 
show prognsss in school, and provided^ -the ability for much better inter- 
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action between the students and teachers, as well as the parents and their 



children's school. The parents were appreciative of their childrens' pro- 
gress in Ehglish acquisition, and expressed their desire to maintain their 
children 1 s Spanish proficiency as well. The parenfevftlsn felt the schools 
had kept than well informed as to student p rogress. All respondents ^ere 
relatively unanimous in their desire to see the program continue. 



Reoatinendations 

1. In the evaluator's opinion, given the data (results) and the favorable 
oannents by participating staff and administrators, and the overwhelming 
favorable perceptions by the participating student's parents, Federal 
funding shculd continue to be sought* and/or alternate means for financing 
the continuation of the Title VII program should be pursued. 

2. Within' the obvious monetary constraints, emphasis should also be placed 

on providing new and updated materials, and instructional aids and devices, 
as mentioned by both staff and school administrators. 

3. Objectives should be revised to more accurately reflect Program emphasis. 
For exarqple, bay*i on the history of the data, and the stated program 
enfiiasis on Ehglish acquisition by LEP students, Objective 1 should be 
revised to reflect different expectations of performance by students 
taking the LAS tests. Different growth parameters would be expected on 
the Ehglish and Spanish portions, and should be more closely equated to 
stated yearly objectives. 

4. Itevise or eliminate Objective 4. The rating of the students' positive 
cul tu ral awareness is to tally subjective, and the device itself does not 
provide partic ular ly iteaningful data, or accurate measurement of the 
desired behavioral attributes. 
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CHAPTER I 



Oklahoma City's Title VII Bilingual Education Program is funded under 
the Federal Elementary and Secondary Education Act (P.L. 92-318) , The pri- 
mary purpose of the Bilingual Education Act has been to establish equal edu- 
cational opportunity for all children, through the establishment and oper- 
ation of educa tion piujiaus using bilingual educational practices, te' iques, 
and methods. S pecifically , the intent is to provide for children of limited 
English proficiency, classroom instruction designed to enable than, while 
using their native language, to achieve competency in the Etoglish language. 

The Oklahoma City Public Schools Bilingual* program began in January of 
1977, in three t ar ge t elenentary schools (Vfestwod, RLvez^ide, and Shidler) , 
which have high concentrations of Spanish speaking students, and was funded 
for $58,462. FVar the current grant period (1980-81) , a fourth school (Ixang- 
ffcMow) has been added to the progr<*n, and the funding level for thess four 
schools for 1980-81 was $115,048. 

Evaluation of the 1980-81 Bilingual Education Title VH Program was 
conducted to both met Federal requirements, and to provide local decision- 
makers with information on which to base any ne ces sa r y decisions concerning 
continuation and/or expansion of the program. Additionally, this evaluation 
is designed to provide information concerning the effectiveness of the 
c urtail program, and to illuminate any necessary upgrading of project 
services. 

/ 



The primary purposes of this evaluation are: 

*Xb meet existing requirements of the United States 
Office of Education related to funding. 

# Tb assess project progress in meeting objectives, 
and jjipLrantaticn of project activities. 

Title VH Program Goals 
The gmeral goal of the Qklatana City Public Schools Bilingual Title 
Vii Program continues to be to provide an effective program of bilingual/ 
rnulticultural education which is designed primarily to increase the level 
of achieveresnt for children of limited Ehglish speaking ability, as well 
as to provide opportunities for student appreciation of cultural pluralisn 
in participating classrooms. 

Tit^e VII Program Objectives 
During the 1980-81 school year, the piujiam objectives , as specified by 
the grant, were designated as follows: 

1. By »y 30, 1981, 70% of limited Bsglish profi- 

ciency stirfencts in grades K-5 wIIl show a gain 
of one standard deviation (based on naming 
group) between pre end posttesting on the 
Language Assessment Scales. 

2. By Itay 15, 1981, 80% of kin d ergarten, first, 

and second grade students will shew gain 
on the Bodm Test of Basic Concepts between 
pre and posttesting in their dominant lan- 
guage* 

3. By May 15, 1981, gain in the mean normal 

curve equivalency (NCE's) score of parti- 
cipating students, as measured by pre 
and posttesting with Prueba de Lectura, 
Serie Intaeraroarica will be greater than 
zero. 
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4* By May 15, 1981, 70% of participating students 
will exhibit positive attifcjdes toward their 
own culture, as well as other cultures repre- 
sented in the classroom, as measured by the 
Cftlahorat city Ifaf erred Cultural Attitude 
Scale , 

Program Description 

The 1980-81 Bilingual Education Title VII Program is an expansion of 
the first year grant (1976-77) . The three original target elementary 
schools - Vfestwood, Shidler, and Riverside - continue to be locations for 
currait grant efforts and a fourth school, Longfellow (5th Year Center) has 
been added far 1980-81. Title VH funds were utilized for 8 of the bilin- 
gual aides in second through the fifth grades, in the four schools during 
1980-81* District funds were utilized for 9*5 Title VII aides in grades K-2. 
In addition to providing necessary teaching materials and staff training, 
Title VH funds financed the 8 bilingual teacher aides, cne program coordi- 
nator, one program resource specialist,, and educational opportunities and 
activities for participants 1 parents. 

Activities conducted by the classroom teachers (and their bilingual 
teach er aides) were designed to enhance participating students 1 oral, read- 
ing, and writing skills in both English and Spanish- Both languages were 
u tiliz ed in creative arts, subject area teaching/tutoring, and cultural 
enrichment activities. 

Evaluation Objectives 
language Proficiency Skills ' 

1. To assess changes in English and Spanish language 
prof icieicy levels of limited English proficiency 
(LEP) students, and ncn-Ehglish speaking students, 
in grades K-4 at Westwood, K-5 at Riverside and 
Shidler, and 5th grade at Longfellow. 



2. Tb ascertain the percentage of li m i t ed Ehglish profi- 

cLmcy and ncn-Ehglish speaking students making a 
gain of one standard deviation (one level) or more 
between pre and posttesting on Language Assessment 
Scales (LAS). 

3, TO conpute potential statistical significance of 

changes in students' proficiency levels. 

Levels of Concept Mastery 

1. TO assess differentials in levels of basic concept mastery 

of participating students in grades K-2, between pre 

and posttesting on the Boehm Test of Basic Concepts (HTBC) . 

2. Tb ascertain the percentage of participating K- 2 st udents 

mking a gain between pre and posttesting on BIBC. 

3. Tb conpute potential statistical significance of changes 

in students 1 basic concept mastery levels. 

Spanish Vocabulary and GOTtcehensicn Reading lave! 

1. Tb assess differentials in mean normal curve equivalency 
ftO!) scores for participating students in grades K-5, 
between pee and posttesting on Prueba de Lectura, Serie 
InLtfTrHitti^ica. 

Cultural Attitudes 

1. Tb assess the number and percent ofc, participating students 
in gr ades K-5 who exhibited positive attitudes toward 
their own and other cultures represented in the class- 
room on pre and/or posttesting on the Oklahoma City 
Inferred Cultural Attitude Scale . 

Staff Training 

1 # Tb assess participating staff's perc ept io n s of in-service 
training provided by the program. 

Adninistratars Assessment of Program 

1. Tb ascertain program needs/strengths/veaknessess, as 
perceived by the appropri ate adninistrators, in the 
target schools. 

Parent Assessment of Program 

1. Tb provide a description of the p rogr a m needs/strengths/ 

weaknesses in effectively axnnunicating with participating 
stoxtants' parents, as perceived by parents of those students. 

. / 
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Design and Methodology 
The following instruments were utilized as measurement devices to deter* 
mine the anoint and direction of changes in participating students 1 Spa n ish 
and Qaglish proficiency skills, mastery of basic concepts, Spanish vocabu- 
lary TOd reading comprehension skills, and cultural atti tudes . 

1. Language Assessment Scales (IAS), Linguanetrics, Inc. 
The £flS tests were administered individually to 
participating students in the Spring of 1981. 
Students' scores on the IAS from the Spring of 
1980 were utilized as "pretest" scores, while 
the Spring, 1981 scores utilized as posttest 
scores. Any student who did not have a pre- 
test score from spring of 1980 was pre te stffd 
in the fall of 1980. Each student (when 
possible) was tested in both Ehglish and 
Spanish. The LAS is a convergent assessment 
measure; it provides an overall measure of 
oral linguistic proficiency, based on students 
^f^f i u nnB across four linguistic 'subsystems 
(phonsnic, referential, syntactical, and 
pragmatic) . Results of the test provide a 
general understanding of linguistic profici- 
ency, relative language proficiency in Ehglish 
ard Spanish, identification of linguistic 
problans, breakdown of linguistic develop- 
ment by subsystan, and a set of prescriptions 
for each student tested. 

Students are ranked by oral production prof i- 
ciency levels (as follows) based on their total 
score test results: 

-Level 1. Minimal Production: production of 
only isolated words, and expressions (54 or below) . 

-Level 2. Fragmented Prortiiction : production of 
a few isolated phrases and fragmented or very 
giirpl* sentences (55 to 64) . 

-Level 3. Labored Production ; production of 
ocraplete sentences with systaratic errors in 
syntax and fact (65 to 74) . 
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-Level 4. Near Perfect Production: production of 
o 5BE5 5t sentences with natlve-uxe fluency and only 
occa aiona] errors in syntax or vocabulary (75 to 84) . 

-Level 5. Perfect Production : production of ccnplete 
sentences which are coherent, syntactically correct 
for developmental age, and reflect accurate repro- 
ductions of the story (85 to 100) . 

Pre and posttest scores were compared to determine whether 70% of the 

K-5 limited English proficiency, and nai-Ehglish speaking students, made a 

gain of one standard deviation (one level) , and whether or not the changes 

in scores were statistically significant (see Results Section for description 

of the par ticular statistical analyses of each component) . 



selected to measure basic understandings of 
con cep ts which are asfwmflfl to be prerequisites 
for language learning and later school success. 
The BXBC was administered as a group test to 
each class (K-2) in the Fall of 1980, and 
again in the Spring of 1981. Students ware 
tested, pre and pott, in their dominant lan- 
guage: Spanish or English. The test is 
divided into 50 items; scares are based on 
number of correct responses to the items. 
Pre and posttest scores were co m pared to 
determine whether 80% of the participating 
students in grades K-2 showed a gain, and whether 
or not the changes in scores were statistically 
significant (see Results Section) . 

3. Prueba de Lectura. Serie Interamerica , Guidance 
— Tes ting As aoHat esT This in stru me nt was selected 
because it was the only one of the Spanish reading 
tests that approached the national noons sp eci f i ert 
by federal regulations. The test measures students 

vocabulary and u uu pren s n ainn reading skills in 

Spanish. Tests ware administered in target schools 
to grades 1-5 in the Fall, 1980. These scores were 
utilized as pretest scores, and the Spring, 1981 
scores used as the posttest. Comparisons ware 
made of the mean pre and post normal curve equival- 
ency scores (NCE's) to determine if those scores 
in each affected grade shewed a gain (see 
Results Section) . 
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4. Qklahana City Inferred Cultural Attitude Scales 

This instrument is a local adaptation of the In- 
ferred Self-CoroeE/' Scale fran Bernard Cohen Research 
and Developngit Car^any, The scales are ccnpleted 
on each K-5 participating student by his/her 
teacher in the Fall of the year (or upon entrance 
to class) , and in the Spring. The instrument con- 
sists of items designed to assess students' atti- 
tudes toward their own cultures and other cultures 
represented in the classroom- The four-point scale 
ranges from "Never" displaying a particular positive 
attitudinal indicator (1) to "Always" displaying 
a positive attitude on the indicator (4) . Total 
scores may range fran 8 to 32, with average scale 
scores ranging fran 1 to 4. An assessment was made 
of the frequency with which positive attitudes were 
displayed among participating students on the pre 
and posttest separately (as the puxjiam objective 
states only that positive attitudes will be exhibited 
by a p erce nta ge of students, with no stipulation 
that there be any diange between pre and post scores) . 

5. Locally Developed End-of -Year Surveys 



strengths and weaknesses of the Bilingual Pro- 
gram, as perceived by staff connected with the 
program, and parents of participating students. 
Open-ended questions were also included to 
cfctain specific input regarding suggested changes 
to be made in the p ro g ram. 




ERLC 



- 7 - 



r 



Results and Discussion 

Objective 1. By May 30, 1981, 70% of limited English proficiency 
students in grades K-5 will show a gain of one stand- 
ard deviation (based on nonning group) between pre 
and posttesting on the Language Assessment Scales 
(LAS). 

In analyzing the performance of Title VU students on the LAS test results, 
several limitations in the data are apparent, and require explanation prior 
to analysis. First of all, the objective requires providing a percentage 
of students who show a gain of one standard deviation (or one "level" , as 
was shown in the description of the LAS test, see page 5) between pre and 
posttesting. These standard deviations (levels) are based on national norming 
data from either monolingual English or Spanish speaking students taking the 
test in t heir dominant language. The (5) levels range from a raw score of 
54 to 100, with all scores lower than 54 Clevel 1) labeled "minimal" pro- 
duction, and scores between 85 and 100 (level 5) labeled "perfect" production. 
The first limitation in analyzing the data, according to the objective as 
written, concerns that group of students whose pretest score is already at 
level 5 (above 85 raw score) . Regardless of the degree of improvement they 
showed between pre and posttesting,' they could not improve by a standard 
deviation, because they started at the highest possible level. Furthermore, 
traditional significance testing on these scores would be very biased, due 
to a "ceiling effect" at that level of responding. Therefore, for the purpose 
of this analysis, very high level initial competence students were not 
included (total of only 19 students in English and 9 in Spanish) . 

A further constraint concerns those students who fall below a raw score 
of 54. A student who scored a 52 on the pretest and a 56 on the posttest 
would be considered as moving up a level. However, a student could score 
a 22 on the pretest and improve, their score to 52 on the posttest and be 
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represented as not making a standard deviation gain. Therefore, the stu- 
dents who scored less than 45 on the pretest will be analyzed separately, 
to determine if they T ^ < ° significant gains, regardless of whether they 
crossed the boundary for Level 2 (•■Fragmented" production) . The raw score 
of 45 was selected because it represents the equivalent of one standard 
deviation (10 points) below the Level 2 boundary. 

. Analysis by Grades 

Kindergarten 

Figure 1 shows the breakdown of students (by percentage) by standard 
deviation far Kindergarten. 

Figure 1 
Kindergarten 
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Figure 1 shows that of those students whose initial coff?3etency was 
within one standard deviation of "minimal " production, 76% showed a one 
standard deviation (level) increase (or. better) in the English portion. 

i 

EVsr the Spanish partial, 40% showed an increase of at least one level* 
The a na lysis of these scares by t-test for correlated samples (repeated 
neasures on. the same individual) , reveals that the students made significant 
gains in both English and Spanish (see ^Figure 1) . 

Tfco points are apparent from these data. First of all, for Kindergarten, 
it is obvious (as would be expected) that more individuals have higher 
initial conpetency in Spanish than in Ehglish. Second, although the stur 
dentfrnwte significant gains iA both languages, the percentage of students 
making a one level jvnp was much higher in English. This probably reflects 
the aiphasis of the Bilingual program on primarily increasing ccnt»tency 
in axjlish for these limited English speaking students. 
Grade 1 

Figure 2 shows the breakdown by percentage of students by standard 
deviation- gain for grade 1. 
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STANDARD DEVIATION GAIN 
GRADE 1 ANALYSIS 



Lang 


N 


Pre x 


Post X 


diff 


^ error 


t-Value 


df 


2-tail Prob. 


English 
Spanish 1 


20 
43 ' 


^56. 65 
59.70 


68.05 
«.42 


-11.40 
- 5.72 


2.94 
1.33 


-3.88 , 
-4.23 


19 
42 


.001* 
.061* 



As Figure Z shows, for grade 1, 70% made one standard deviation gain' 
or better in English, while 54% made one standard deviation gain or better 
in Spanish. Again, the t-test for correlated sample showed significant 
gains were made in both languages. These results are consistent with Kinder- 



Grade 2 



Figure 3 show the breakdown of percentage of students by standard 
deviation gain far Grade 2. 
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Figure 3 
Grade 2 
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STANDARD DEVIATION GAIN 
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GRADE 2 ANALYSIS 





N 


Pre x 


Post X 


diff 


S error 


t-Value 


, df 


2-tail Prob. 


tajlish 


49 


63.35 


75.06 


-11.71 


1.20 


-9.78 


48 


, .001* 




"70 


57.94 




- d.23 




-0.21 


59 


.83 
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Figure 3 shows that far grade 2, 82% made one standard deviation gain 
or better in English, vfeile 31% made one standard deviation gain or better in 
Spanish. This is a Lower p e r ce nta ge than previous grades for Spanish and 
the tr-tast reflects that the mean gain far Spanish between pre and posttest- 
ing in 2nd grade, was not significant. 
Grade 3 

Figure 4 gives the breakdown of percentage of students by standard 
deviation gain for grade 3. 

Figure 4 
Grade 3 




STANDARD DEVIATION GAIN 
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/ 
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GRADE 3 ANALYSES 



Iang 


N 


Pre x 


Post X 


. diff 


S error 


t-Value 


df 


2-tail Prob. 


ajqliah 
■Spanish 


26 
18 


69.81 
^.50 J 


( 

78.92 
S9.89 


-9.11 
-4.29 


2.00 

1.55 


-4.55 


25 

"37 


.001* 
.009* 



Figure 4 stows that for grade 3, 61% made ens standard deviation 
gain or better in Eftglish, while 55%Ttedecne standard deviation gain or 
better in Spanish. Cnce again, the t-test demonstrates that, significant mean 
gains were made in both languages. 
4 
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Figure 5 gives the breakdown of percentage of students by standard 
deviation gain far Grade 4. 
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GRADE 4 ANALYSIS 





N 


Pre x 


Post X 


diff 


S error 


t-Value 


df 


2-tail Prob. 


Owlish 


30 


70.4 


84.33 


-13.93 


1.40 . 


-9.94 


29 


.001* 




15" 


68.82 


'76.11 


- 7.29 " 


1.32 


-5.54 


44 


.001* 



Figure 5 shews that for grade 4, 93% made one standard deviation gain 
or better in English, and 52% made one standard deviation gain or better in 
Spa ni sh Again the t-test shows significant mean gains were produc ed in 
bo th . 
Grade 5 

Figure 6 gives the breakdown by percentage of students by standard 
deviation gain far Grade 5. 

Figure 6 
Grade 5 * 




I ENGUSH 
» SPANISH 



ERIC 



STANDARD DEVIATION GAIN 
-15 - 



GRADE 5 ANALYSIS 



Lang 


N 


Pre x 


Post. X 


diff 


S error 


t-Value 


df 


2-tail Prob. 


English 
Spanish 


15 
21 


70.33 
71.36 


80.00 
■ 72.71 


-9.67 
- .81 


1.55 
1.93 


-6.22 
-6.42 


14 
26 


.001* 
.68 



Figure 6 shows that for grade 5, 87% irade cne standard deviation gain 
in English, while 33% node a one standard deviation gain in Spanish. As with 
the 2nd grade, the t-test reveals that the mean Spanish gains for 5th gra^e 

were not significant. 

Stmnary By Grades 

Table 1 shows the percent by grade, and by language, of students making 
a one standard deviation (level) gain on the LAS. 



TABLE 1 
LAS Results By Grades 



Grade 


Language 


N 


% 1 Sd or greater 


K 


English 
Spanish 


17 
47 


76% 
40% 


1 . 


English 
Spanish 


20 
43 


70% 
54% 


2 


English 
Spanish 


43 
70 


82% 
31% 


3 


English 
Spanish 


26 
38 


61% 
55% 


4 


English 
Spanish 


36 
45 


93% 
52% 


5 


English 


15 
21 


87% 
33% 


Total 


English 
Spanish 


157 
264 


78.98% 
43.56% 



- 16 - 



As Table 1 shows, of those students who started within one standard 
deviation of the Level 1 "minimal" production boundary (raw soore=54) , almost 
79% made a one standard deviation gain (one level) in English. Of those 
students who pre-tested within qne standard deviation of ''minimi" pro* 
ducticn in Spanish, approximately 44% made a one standard deviation gain 
(one level) . A t-test for correlated sanples (repeated measures on the sane 
individual) for each grade revealed significant (p< .002) gains were made 
at every grade level for English, and significant gains (p«c .009) were also 
made far every grade (except grades 2 and 5) in Spanish. 

Several points are illuminated by these data. First, as would be ex- 
pected, more individuals in each grade pretest at least within one SD of 
"minimal" production in Spanish as opposed to Biglish. Given the ^jopulation 
in the Title VH program this is not surprising. Second, even though signi- 
ficant learning gains are apparen in both Ehgl ish and Spanish, a far greater 
percaitage of students (79% vs. 44%) made at least: a one level (SD) gain in 
English. This appears to reflect the anphasis of the Title VII program, in 
teaching these limited Ehglish ability students to function in primarily 
Ehglish speaking (yet preserving their bilingual capabilities) - 

Therefore, although the data tend to appear to reflect this enphasxs, no 
provision for separation exists in the stated objective per se, so in essence, 
the objective as stated was only half met. 

Low IhiH^I - Competency St**^*** - As was mentioned earlier, a portion 
of the students in grades K-5 pre-tested at least one standard deviation 
below tte "minimal" production level (less than 45 raw score) . Table 2 
gives the breakdown by grade, and by language, of these students, and also 
represents the t-test for correlated sanples (repeated measures) conducted 



to determine if significant progress occurred, regardless of whether they 
made it to the 2nd Level ("Fravnented" production) . 

TABLE 2 

Low Initial Coipetency Students 



Grade 


Language 


N 


Pre x 


Post X 


diff 


.Serror 


t-Value 


df 


2-tail Prob. 


K 


English 
Spanish 


35 
4 


25.77 
37.50 


53.80 
57.75 


-28.03 
-20.25 


1.829 
7.122 


-15.32 
- 2.84 


34 
3 


.001* 
.06 


1 


English 
Spanish 


29 
5 


26.72 
34.80 


51.52 
42.40 


-24.8 
- 7.60 


2.42 
6.82 


-10.24 
- 1.11 


28 
4 


.001* 
.33 


2 


English 
Spanish 


25 
4 


21.40 
27.00 


47.04 
43.00 


-25.64 
-16.00 


2.96 
3.37 


- 8.67 

- 4.75 


24 
3 


.001* 
.018* 


3 


English 
Spanish 


13 
5 


21.08 
17.40 


41.23 
39.80 


-20.15 
-22.4 


3.05 
2.82 


- 6.61 

- 7.94 


12 
4 


.001* 
.001* 


4 


English . 
Spanish 


13 
2 


28.62 
31.00 


46.54 
53.00 


-17.92 
-22.00 


3.98 
1.00 


- 4.51 
-22.00 


12 
1 


.001* 
.029* 


5 


English 
Spanish 


8 
6 


25.75 
27.67 


47.13 
34.67 


-21.38 
- 7.00 


3.17 
4.34 


- 6.74 

- 1.61 


7 
5 


.001* 
.17 ' 



Table 2 reveals several points of interest concerning these low initial 
coipetency students. First, as would be expected, the proportion by language 
is exactly opposite of the previous analysis. Far more students have very 
low initial competence in English than Spanish. Howevor, consistent with 
the analysis to date, these students made highly significant gains in English 
in every grade (p ^.001) . The non-significant gains in grades K, 1, and 5, 
for Spanish are nost probably «a artifact of extremely -nail sample size, 
and/or large standard errors. 

The pre-to-post means are also noteworthy. Note that for English, 
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the mean gains between pre and posttesting range from 17.9 to 28.0 (raw 
score) . These are very substantial gains. Without the ability to oatpare 
t hese scores to a no-treatment control group, it is difficult to attribute 
all the gain to the program treatment effect. However, this magnitude of 
raw score gains is very unlikely (p <..00I) to have occured by chance, and 
when co n s ider ed with the previous srrulysis, which showed significant English 
gains at all grades for the more initally competent students, does tend 
to enjAasize that these limited Ehglish proficiency students iitnroved their 
English carprehensicn substantially during this school year relative to 
their starting position. Further, roost of than inproved their Spanish 
corprehension as well. 

Another note of interest concerns the descending magnitude of initial 
low carpetency students in English. The mmber per grade decreases from 35 
in Kind e rg art en to 8 in the 5th grade and may veil be a reflection of pre- 
vious Title VH efforts in past years. This seems intuitive, yet cannot be 
proved with this data. 

Objective 2- By May 15, 1981, 80% of kindergarten, first, 
and second grade students will shew gain 
on the Boehm Test of Basic Concepts between 
pre and posttesting in their dotanant lan-r 
guage. 

As rentioned earlier, the BOEHM measures mastery of basic concepts, 
viiich are presuned to be prerequisties for language learning, and therefore 
predictors of later school success. Each K-2 student took the test in their 
<±mnant language. The results by grade are shewn in Table 3. 



TABLE 3 
BOEHM Results by Grade 



Grade 


N 


Number of 

S's showing qain (%) 


Nunber of 
S's same (%) 


Nunber of 

S's decrease (%) 


K 


55 


52 (94.5%) 


2 (3.6%) 


1 ( 1.9%) 


1 


64 


53 (82.8%) 


3 (4.7%) 


8 (12.5%) 


2 


88 


61 (69.3%) 


8 (9.1%) 


19 (21.6%) 


Total 


207 


166 (80.2%) 


13 (6.3%) 


28 (13.5%) 



As Table 3 shews, even though there were fluctuations by grade, 80.2% of 
all s tuden ts in grades K-2 made gains between pre and posttesting, 

while 6.3% retained the same, and 13.5% showed a drop. 

Table 4 gives the analysis by grade, and by doininant language, of whether 
the nean gains of the students in each grade were significant, again, a t-test 
for correlated samples (repeated measures) was utilized. 



1AHE£ 4 

S 

BOEHM Gains By Grade and Dominant Language 



Grade 


Language 


N 


Pre Mean 


Post-Mean 


Diff 


S error 


t-Value 


df 


2-tail 
Prob. 


K 


English 
Spanish 


17 
38 


29.41 
26.16 


40.53 
35.92 


-11.12 
- 9.76 


2.07 
1.34 


-5.37 
-7.28 


16 
37 


.001* 
.001* 


1 


English 
Spanish 


20 
44 


36.0 
33.59 


42.40 
38.48 


- 6.40 

- 4.89 


1.13 
0.76 


-5.67 
-6.42 


19 
43 


.001* 
.001* 


2 


English 
Spanish 


15 
73 


44.20 
41.08 


46.33 
43.79 


- 2.13 

- 2.71 


0.88 
0.55 


-2.42 
-4.94 


14 
72 


.03* 
.001* 



As Table 4 shows, for each grade, regardless of which language the stu- 
dents we \b tested in, -significant gains were made between pre and posttesting. 
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Objective 3. By May 15, 1981, gain in the mean normal 
curve equivalency (NCE's) score of parti- 
cipating students, as measured by pre and 
posttesting with Prueba de Lectura, Serie 
Lateranarica will be greater than zero. 

Tte Prnftba de Lectura, Serie Interaroarica measures students' vocabulary 

and ocnprekansiop reading skills in Spanish. National Nanns are provided for 

each testing dates (Fall and Spring) , and students raw scores are translated 

into *wnnal curve equivalency scares (NCE's) . It should be enphasized that 

a mean NCE gain of zero between pre and posttesting does not imply that no 

learning has occurred. In fact, an average NCE gain of zero means that the 

studeit has maintained the pace of the national norrning group for the year, 

r elat ive to their pretest starting position. Any mean NCE gain greater than 

zero means the learning gains have exceeded the national nam at that grade 

level. Oily s tuden ts with botn pre and pcsttest scores are included in this 

analysis. 

Table 5 shews the breakdown by gradts of the students 1 performance on 
the Prueba de Lectura. 



TABLE 5 

Prueba de Lrctura Gains by Grade 





N 


Mean Pre NCE 


Mean Post NCE 


Mean NCE gain 


1 


36 


51.36 


51.83 


40.47 


2 


46 


50.51 


54.56 


+4.05 


3 


59 


50.3] 


50.04 


-0.27 


4 


50 


49.65 


51.10 


+1.45 


5 


19 


51.01 


52.03 


+1.02 


Total 


210 


50.57 


51.91 


+1.34 



As Table 5 shows, for all grades (with the exception of grade 3 where 
a slight NCE drop occurred) there were NCE gains. Further, as a total, 
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the mean NCE gain was greater than zero. All the normal curve equivalencies 
are at or about the 50th percentile, and the NCE gain of greater than zero 
indicates the students' progress in vocabulary and apprehension of Spanish 
progressed relatively equal to, or slightly superior to the National nonning 
sample* 

Objective 4 . By May 15, 1981 r 70% of participating students 
will exhibit positive attitudes toward their 
own culture, as well as other cultures repre- 
sented in the classroom, as measured by the 
Oklahoma City Inferred Cultural Attitude Scale. 

A total of 432 stixients in grades K-5 were scored on the Attitude Scale 
by their teachers. As mentioned earlier, the four point scale ranges in - 
values from "never" displaying a particular positive attitudinal indicator 
(1) , to "always" displaying a positive attitude (4) • There are eight situ- 
ations in which each student is rated, and the total scores may range from 
8 to 32 (with the average scores between 1 and 4) . Aa mean score cf 1-2.0 
is considered a "negative" attitude, 2.1-3-0 "neutral", and 3.1 to 4.0 
"positive". Table 6 shows the ratings by teachers for the fall and spring. 



TABLE 6 

Cklahona City Inferred Cultural Attitude Scale 



Dates 


N 


% rated Positive 


% rated Neutral 


% rated Negative 


Fall 1980 
Spring 1981 


432 
432 


84.5% 
87.5% 


14.1% 
12.5% 


1.4% 
0% 

1 r ' 



Stmnary 

• The IAS test was utilized to represent the gains in English and Spanish 
oral ling uistic proficiency by participating students in grades K-5. Pre- 
test results revealed, as would be expected, that a ituch larger percentage 



of students began the year with lower competency levels in English than 
Spanish. Within the constraints noted, the objective of raising 70% of 
these LEP students oral proficiency in Ehglish was met, as aporoximately 79% 
made a one level (standard deviation) gain or better during the year. In 
analyzing the gains of the very low aoroetency^tudents during the year, -highly 
significant (p < .001) gains were noted at each grade level (K-5) , as well 
as substantially larger raw score mean gains than the overall population. 
In all grades (K-5) , the mean gains in English oral proficiency were highly 
significant (p-c.002) , for all groups of students, ranging from extreme 
"iramiral" production groups to "near perfect" produ c t io n groups (group lahela 
represent "level" distinctions as specified by the IAS test norms) . 

The same students' performance on LAS Spanish oral proficiency was not 
as substantial as their English performance, yet in most cases (and most 
grades) produced significant gains. Within limitations, approximately 44% 
(less than the 70% specified by Objective 1) of the students showed a one 
level (standard deviation) gain or better during the year. Basically, only 
at grade 2 and grade 5 did they not show significant (at least p< .02) mean 
gains between pre and posttesting. This lower percentage of gains in Spanish 
appears to be a reflection of the enphasis of the Title VII p ro g r am toward 
pr om oti ng English acquistion by these LEP students, in an attempt to make 
them nore able to carpete and excel in English speaking classrooms. It is 
important to note, that alttough the students did not improve as dramatically 
on the Spanish portion of the LAS as they did on the English portion, sub- 
stantial gains in Spanish proficiency were reported. This is supported by 
the data from the Pr¥*m da Lectura , Serie Interamerica results, which 
showed that students 1 vocabulary and co i pr ehensicn redding skills gains in 
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Spanish, were equal to or higher than the national norming samples for almost 
every grade. For each grade, first through fifth (except 3rd grade, which had 
a slight NCE drop) the NCE gains were greater than zero, and overall, for all 
students, across all grades, showed a NCE gain of +1.34 (meeting Objective 
3). Any NCE gain greater than zero indicates that progress was made in ex- 
cess of predicted progress based on national norms for the year. 

For the early elementary students (grades K-2) , the results of the Boehm 
Test of Basic Concepts snowed that overall, approximately 80% made gains 
during the year thereby meeting Objective 2. The BTBC reportedly measures 
acquistion of basic concepts, which are presumed to be prerequisites for 
language learning, and therefore predictors of later school success. Further, 
regardless of whether the students dcminant language was English or Spanish, 
the mean raw score gains in each grade (K-2) were highly significant (most 
less than p<.001) for the year. 

Finally, although the teacher ratings are totally subjective they in- 
dicate the Objective 4 was met, since the Bilingual teachers and aides rated 
87.5% of the Title VII students as having a positive attitude toward their 
own, as well as the other cultures represented in their classrooms (an 
increase of 3% over their rating at the start of the year) . 

Participants' Perceptions of the Program 
School Administrators. The principals at the school sites all reported 
they had attended inservice training activities presen t ed by the Title VII 
program. All believed that the Title VTI staff made an effort to provide 
information about the purposes and services available through th? program, 
and t heir Title VII building staff regularly attempted to transmit infor- 



- 24 - 



nation to them concerning problems or progress in working with the students. 
They ^1 <*^ felt that the aides were invaluable in providing an interface be- 
tween the staff and the Spanish speaking students, and their Spanish speaking 
parents. As far as reoccmendatidhs for improvement, they felt the program 
should be continued, with enphasis an new and updated materials, and further 
inservice and training for teachers and aides at the building level. 

Parents . A total of 102 parents from the four target schools res p o n ded 
to tiie Title VH parent survey. Of these 102, 91% stated they had received 
information (in both Spanish and English) throughout the year, explaining 
both the purpose and goals of the Oklahoma City Public Schools Bilingual Edu- 
cation P r o gram (see Appendix A for description of the survey form, and specific 
percentages of responses to each item)*. Further, between 81-94% responded 
that they had been notified by their childrens 1 school in Ehglish and Spanish 
of occ ur r e nces such as report cards, parent cenferenoe reports, notes from 
school, necessary immunization information, and parent carmittee meetings. 

The vast najority (91%) of the parents felt that their child had bene- 
fited fran the Bilingual Education program, and they felt it was important 
that t heir c hildr en were being taught both English and Spanish, as they 
wanted their children to continue their bilingual cap abilities; Most of 
the parents volunteered that they had witnessed i mpr o v ements in their childrens' 
progress in school, and are pleased wilh the program. A large nutter of them 
alao <xmn tfnto fl that they were very grateful for the schools 1 concern and un- 
derstanding, and appreciated receiving needed information in Spanish. 

As for recorormding changes to improve the Bilingual E d u cation services, 
all of the parents who responded indicated they wanted the progress to con- 
tinue. A representative statement was they feel the children are making 

- 25 - 

37 



progress, and the Bilingual piuguu enables the parents to ocnmunicate with 
the school. 

None of the parents stated the pimji a u should be discontinued, and only 
2 of the 102 suggested that only English be taught. No other recamtendations 
for improvement were noted. 

Staff training and perceptions . The Title VII staff attended inservice 
training and workshops throughout the year, ranging in duration from 1-2 days, 
to a 2-week pre-school workshop. Between 75-100% of the participants on each 
occasion, rated the quality of the workshop good to very good. Roughly the 
sane percentages felt the emphasis of the workshops were directly applicable 
to their work, and most materials and topics were relevant and necessary. 
The majority of individuals reported each -time that more training and assist- 
ance would be useful, and suggested future topic areas included, (1) more 
training to teach Spanish, (2) introduction of new materials, (3) more 
application of ESL in science, math and literature, (4) stress management, 
(5) more training in arts, crafts, and games, and (6) more samples of visual 
presentations for ESL. (Full workshop evaluation summaries for the various 
inservice training and workshop sessions and available upon request.) 

Parent English Classes. A final aspect of the Bilingual program invol- 
ved ESL training for parents of Title VH students, weekly at the Capitol 
Hill Library. The classes were conducted in the evening, and approximately 
50 parents were "regular" participants. Thirty-two parents returned the 
Title VH Bilingual Education Parent English Classes Survey (see Appendix 
' a for percentage of responses to each question) . Of the 32, 87% had attended 
the classes more than three times, and alirost 60% of those more than 5 times. 
Tte parents responded they basically liked everything about the classes, 



reported nothing was unsatisfactory, and wanted to continue. The majority 
of than (80%) felt the classes ted helped them in their everyday aannunica- 
tions, and had helped them in their jobs. Most felt they needed to learn 
more. 

Siamary * The vast majority of responses by. all participants and parents 
in the surveys were favorable. School administrators , staff, and parents 
felt the program was beneficial to the students , enabled them to shew pro- 
gress in school, and provided the ability for much better interaction be- 
tween the students and teachers, as well as the parents and their children's 
school. Hie parents were appreciative of their children's progr ess in 
English acquisition, and*expressed their desire to maintain their children's 
Spanish proficiency as well. The parents also felt the schools had kept 
than well infomed as to student progress. All respondents were, relatively 
unaninDus in t heir desire to see the program continue. 
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R eu uiinendations 



In the ©valuator's opinion, given the data (results) and the favorable 
commits by participating staff and administrators, and the over- 
whelming favorable perceptions by the participating student's parents, 
Federal funding should continue to be sought, and/or alternate meens 
for financing the continuation of the Title VTE program should 
be pursued. 

Within the obvious monetary constraints, emphasis should also be 
placed on providing new and updated materials, and instructional aid* 
anddeyices, as mentioned by both staff and school administrators. 

Obiectives should be revised to more accurately reflect Program 
emphasis. For example, based on the history of the data, and the 
stated program enphasis on English acquisition by LEP students, 
Objective 1 should be revised to reflect different expectations 
of performance by students takir*? the LAS tests. Different growth 
parameters would be expected on the English and Spanish portions, 
and should be more closely equated to stated yearly objectives. 

Revise or eliminate Objective 4. The rating of the students' 
positive cultural awareness is totally subjective, and the device 
itself does not provide particularly meaningful data, or accurate 
measurement of the desired behavioral attributes. 
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OKLAHOMA CITY PUBLIC SCHOOLS 
PLANNING, RESEARCH, AND EVALUATION DEPARTMENT 



BILINGUAL TITLE VII PROGRAM 
PARENT QUESTIONNAIRE 
1980-81 



1. Have you received any infor- 
mation explaining the purpose 
and goals of the Oklahoma 
City Public Schools Bilingual 
Program? 



U^ 63 2.9ffl o 5.9% 



2. Has your child's school com- 
municated to you in English 
and Spanish in any of the 
following manners: (check any 
or all as appropriate) 



94.1% teport Card 

93 . lfe arent Conference 
Reports 

84«3% jotes From School 

81 . 4Zi nmiuni2ation 
Information 

83.3ZP arent Committee 
Meeting 



Other: 



3. Do you feel your child has 
benefited from the Bilingual 

Education Program? 9 JL*F e s 5 ^I*J*o 2.91 



Please explain: 
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4. Please suggest any ways you think the Bilingual Program could be made more 
effective fcr you or your child. 
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OKLAHOMA CITY PUBLIC SCHOOLS 
PLANNING, RESEARCH, AND EVALUATION DEPARTMENT 

TITLE VII BILINGUAL EDUCATION 
PARENT ENGLISH CLASSES SURVEY 
1980-81 



We would like you to answer the following questions so we can determine 
how we have done in meeting your needs for English instruction, and what we can 
do to improve services to you in the future. Please return the completed form 
in the enclosed self -addressed and stamped envelope. Thank you for your help. 



How many times did you attend the 
parent English classes at Capitol 
Hill Library? 



CHECK ONE 
3.1% O nce 3.1% No Resp. 
6.2% T wice 

2 8.1% T hree to Five Times 
59.4% More than Five Times 



2. What area of English instruction 
would you like mosc to see covered 
in the classes? 



1 1 .21 R eading in general 

4 3.7% W riting, in general 

7 8.1% S peaking, in general 

21.9% Applying for jobs 

3K±%JDealing with cold's 
school and teacher 

Other (what?) 



ERLC 
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What did you like most about the English classes you attended? 



Was anything unsatisfactory about the English classes you attended? 



Is there anything you would like to see added or changed in the English 
classes? 



Do you feel this class helped you in your everyday cotnnunications with 
others? 

21.91 - Yes, helped "a lot" in everyday communications and at their job. 

59. 4Z - Yes, helped them. 

18. 7Z - Yes, helped them but need to learn more, 



• 34 - 



